Storage technology is at the heart of modern data centers and Big Data applications. Storage performance and manageability continue to be important concerns for storage systems and in particular to the total cost of ownership of such systems. Recent improvements in storage devices using NAND flash memory, such as solid-state drives (SSDs) have resulted in significant changes to storage architectures. Currently, such devices are used in different forms in most storage systems as a fast storage tier. Storage device technology will continue to affect storage systems and architectures in a profound manner. There are two aspects of existing and emerging fast storage devices that are not handled properly by currently known systems: performance and heterogeneity.
Regarding performance, known storage systems are typically designed to operate in the millisecond-range. Even with SSDs, typical storage input/output (I/O) latencies are in the order of hundreds of microseconds, which allows the overhead of the systems software that provides the required storage abstraction and manageability to be hidden. However, as storage device performance begins to improve as projected, with device latencies in the order of a few microseconds, e.g. in the lops range, current approaches to designing storage systems will not scale with the performance (latency and I/O operation rate) of such devices. Systems software and storage protocol overhead are the main bottleneck when it comes to performance.
Current systems are also insufficient when it comes to handling heterogeneous networks, that is, a network connecting computers and other devices with different operating systems and/or different protocols. As device performance improves, storage systems will be required to cope with different types of devices, at least SSDs and magnetic hard disk drives (HDDs). Current storage systems are designed to operate in a different manner when accessing SSDs or HDDs, where for instance SSDs are organized as a separate cache and the path for I/O operations is fixed. Similarly, HDDs are typically combined in RAID (Redundant Array of Independent Disks) arrays that require devices with similar characteristics. These restrictions, which require the careful organization of devices into tiers or arrays within a storage system, create complex and inflexible configurations that are hard to evolve over time and introduce significant management overhead.